Parallel Computing for Large-Scale Optimization Problems: Challenges and Solutions

Optimization refers to the minimization (or maximization) of an objective function of several decision variables that have to satisfy specified constraints. There are many applications of optimization. One example is the portfolio optimization problem where we seek the best way to invest some capital in a set of n assets. The constraints might represent a limit on the budget (i.e., a limit on the total amount to be invested), the requirement that investments are nonnegative (assuming short positions are not allowed), and a minimum acceptable value of expected return for the whole portfolio. The objective or cost function might be a measure of the overall risk or variance of the portfolio return. In this case, the optimization problem corresponds to choosing a portfolio allocation that minimizes risk, among all possible allocations that meet the firm requirements. Another example is production planning and inventory: the problem is to determine the optimal amount to produce in each month so that demand is met while the total cost of production and inventory is maintained without shortages. In recent years the Interior Point Method (IPM) has became a dominant choice for solving large optimization problems for many scientific, engineering, and commercial applications. Two reasons for the success of the IPM are its good scalability on existing multiprocessor systems with a small number of processors and its potential to deliver a scalable performance on systems with a large number of processors. IPM spends most of its runtime in several important sparse linear algebra kernels. The scalability of these kernels depends on several key factors such as problem size, problem sparsity, and problem structure. This paper describes the computational kernels that are the building blocks of IPM, and we explain the different sources of parallelism in sparse parallel linear solvers, the dominant computation of IPM. We analyze the scalability and performance of two important optimization workloads for solving linear and quadratic programming problems.